p-value

2022-03-09 ยท 2 min read

    see: https://www.wikiwand.com/en/P-value

    $$ \renewcommand{\Pr}[1]{\text{Pr}\left[ #1 \right]} \def\eps{\varepsilon} \def\E{\mathbb{E}} \def\pv{p\text{-value}} $$

    Overview #

    • The p-value is a r.v.
    • For a given observation, it's the probability of obtaining observing results at least as "extreme" as the currently observed result, given the null hypothesis is correct.
    • Imagine $f(x)$ is some criterion r.v. which returns the probability of finding more extreme results than some observation $x$. In this case, the null hypothesis is also a distribution which we're assuming $x$ is sampled from.

    $$ \pv(x) = p \in [0, 1] ;:; \Pr{f(x) \ge p ;|; x \sim H_0} $$

    • Used in null-hypothesis significance testing.
    • Different from the transposed conditional probability

    $$ \times ;; \Pr{x \sim H_0 ;|; f(x) \ge \pv} $$

    $$ \begin{align*} \Pr{x \sim H_0 ;|; f(x) \ge \pv} &= \Pr{f(x) \ge \pv ;|; x \sim H_0} \cdot \frac{\Pr{f(x) \ge \pv}}{\Pr{x \sim H_0}} \ \end{align*} $$

    Traps #

    1. p-value is conditional probability, not probability the null-hypothesis is true nor probability alternative-hypothesis is false.
    2. p-value is only statement about the relation of the observed data to the null-hypothesis.
    3. p-value significance of 0.05 is only convention.
    4. p-value doesn't indicate size or importance of an effect.